Super Audio/Super Audio.iso/replay/_armovie/documents/codecif

Text File | 1997-11-10 | 26KB | 552 lines

            The video codec interfaces for Acorn Replay
            ===========================================

Introduction
------------

The Replay "ARMovie" format allows for different sound and video compression
formats. Interfaces inside the ARMovie Player program ("!ARMovie.Player")
allow introduction of new video decompression code, an appropriate
compression program and allow other programs to use the ARMovie video
decompressors. Sound format 2 provides similar independence for sound
compression methods: the format directly names the sound decompression code
which will be used - provided this is located in <ARMovie$SoundDir> (see
ToUseSound for more details).

There are two ways to introduce new video compression formats: total
replacement of the Player program (provide a !ARMovie.Decomp<N>.!RunImage
for a new compression type <N> which implements the entire ProgIf command
interface) or provide a decompressor which uses the Player interface
(provide a !ARMovie.Decomp<N>.Decompress file). A batch compression
interface is defined for compression programs which are named
!ARMovie.Decomp<N>.BatchComp and will be called automatically by !AREncode
(and, presumably, other utilities).

Total replacement decompression interface
=========================================

When a Replay ARMovie file is to be played, the resource !ARMovie.Player
gets called with a command string by the operating system. Player would
normally use the software decompression interface and paint the movie on the
screen itself, but it can also relinquish the entire task (e.g. for a format
which is decompressed by hardware and video overlayed onto the screen) if
that is required. After a minimal check that the ARMovie file is valid,
!ARMovie.Player reads the video format number and checks the appropriate
Decomp directory for the existence of a '!RunImage' file. If this exists,
Player will run it with the ARMovie file and other arguments (many of them
are in ARMovie$ExtraArgs) which were passed to Player.

The replacement routine should obey as much of the command set of Player as
possible (though -shape may be asking too much in the above example!). The
replacement program is free to get its data from anywhere, but it is
recommended that ARMovie files are present with standard headers in order
that general multimedia programs will see something they understand.

Video decompression interface
=============================

The total replacement of !ARMovie.Player is a very large task; it is much
easier to provide a new video decompressor only and rely on the
!ARMovie.Player program doing the rest of the work. Acorn currently provide
10 video decompressors using this interface and many of them are provided
with source.

Using this interface means keeping most of the architecture of the Player
system:

- interleaved video+sound chunk format
- format of sound code and interface to it
- key frame system (but it can be ignored completely, also)
- decompressor called in IRQ mode; should run with interrupts (mostly)
  enabled
- flags between disc loader and decompressor
- flags between decompressor and screen painter
- decompressor provides the colour looked up output

Details of using the sound system are in 'ToUseSound': either the current
Acorn Replay sound code can be used or new sound formats added by sound code
implementing the current interface (described from the point of view of the
client in ToUseSound) can be added using format 2 to name the sound code. No
changes to Player programs is required.

In order to start some of the decompressors (MovingLines, for example) at
any position, the use of key frames is possible. The compressor stores the
state of the stream at the start of each chunk, uncompressed, in the key
frame data structure. The Player program will load the uncompressed file
from the data structure in order to start decompression at an arbitrary
point, providing the 'previous frame' state (see below). Decompressors which
can start anywhere use an ARMovie file without the key frame data structure
(by giving a -1 offset to the keys in the header).

The decompressor is called in IRQ mode with interrupts enabled in order to
decompress the next frame. Code in the Player handles semaphores between
decompression and painting frames and between loading the video chunks and
decompressing frames. Other code in the Player takes data from the
decompressed frames buffer and paints it onto the screen (in IRQ mode also)
and code waits for compressed data buffers to be empty and refills them from
the filing system (in User mode as the foreground task). To allow these
other tasks to run, the IRQ code must leave interrupts enabled for as much
time as possible (the MovingLines decompressor leaves interrupts enabled for
all of its work). In any case IRQ must not be disabled for longer than 1
full sound buffer: about 2mS.

The disc loader works in a double buffered manner: two buffers each
containing either one or two chunks of data (2 or 4 seconds per buffer
conventionally). The addresses of the two buffers and one or two chunks per
buffer are all chosen dynamically and the Player program knows from the
ARMovie header how many times to call the decompression code. The two
buffers will always start on word addresses.

Using the interface play with existing Decompress code
------------------------------------------------------

As well as being able to provide new compression formats by using the
Player/ Decompress interface, other programs can also use the Decompress
code, in effect pretending to be the Player program as far as the Decompress
code is concerned. This greatly extends the capability of the Replay system:
programs could be written to play ARMovie files in the desktop while
multitasking (obviously non-real time). Or sections of an ARMovie file could
be decompressed to memory and edited. Or... The ability to patch the
Decompress code allows player programs to load two copies of a Decompress
code: patch one to generate data for one format and patch the other to
generate data for a different format (e.g. non-dithered for the screen).

Such a playing program has to do all the tasks normally done by the
!ARMovie.Player program:

- read the file header to discover the data
- establish the dither for the screen if needed
- load the right decompress code, patch and initialise it
- load the key frame if required
- load the chunks from the file
- call the decompress entry point for each frame in the block

but it need do no more than this. It can structure its data structures as it
wishes (excepting that Decompress codes write pixels as words).

The Acorn provided BatchComp programs use ARMovie files as input and use the
Decompress code in order to convert the data into a form they understand
(indeed, they will use Decompress code even if the !RunImage total takeover
exists) and thus get total isolation from the video compression method of
the input ARMovie file.

Decompress "Info" file
----------------------

In order that Decompress code providers can incorporate their own names for
formats, provide format origination text, provide compress and edit routines
with hints on various formats, an Info file called Decomp<N>.Info can be
provided (the !ARMovie.Player does not need this file). It contains the
following information:

Name of format
Originator
exact number of bits per pixel (or empty if this is variable)
desired x multiple; minimum x size; maximum x size
desired y multiple; minimum y size; maximum y size
temporal (or empty if temporal information is not required)
list of colour spaces which can be held in the format

For example:

Moving Lines
© Acorn Computers 1993

1;1;1280
1;1;1024
Temporal
YUV 5,5,5; RGB 5,5,5

The colour space information allows programs like !AREncode to present the
user with valid options.

Formal interface: "Decompress" file (any file type can be used)
---------------------------------------------------------------

The Player program loads the decompress code at a quad word aligned address
in memory. It then processes the patch table and then calls the init point.
Later on the player calls the decompressor in USR mode to decompress the
first chunk's worth of frames (or to do -explode). For a full play, the
player will call the decompressor in IRQ mode.

First word: patch table offset
(32 bit word at byte offset 0)

 Purpose: to allow the player to insert pixel colour lookup in the
          decompressor. May in future allow other values to patched.
          Note that an unpatched decompressor should still work!!

 The offset table consists of words. Each word has the bottom 16 bits as the
 byte offset of a word (usually an instruction) from the start of the
 decompressor. The top 4 bits are an opcode number. The remaining 12
 bits may have a meaning for that opcode. The list is terminated by a
 word of -1.

 Opcode 0: patch in colour lookup
  bits 27..24: destination register
  bits 23..20: source register
  bits 19..17: pixel lookup table register
 The Player alters the word to lookup the pixel colour. The value of
 the source register and the size of the result in the destination
 register may change for different decompressors. For MovingLines the
 source register value is either RGB or YUV 15 bits and the destination
 value is always a word value. An unpatched MovingLines decompressor
 produces RGB or YUV output.

Second word: init entry point
(byte offset 4)

 Purpose: to allow the decompressor to initialise any tables or other
          values that are needed. These will be inside the Decompress
          code space or allocated by the init routine.

 On entry:
   r0 - source x size of movie
   r1 - source y size of movie
   r2..r12 - undefined: scratch
   r13 - stack
   r14 - return address
   processor mode: USR
 On exit
   r0-r12 may be corrupted
   r13 - same as on entry
   flags - V set signals error: r0 points to an error block

Third word: decompress entry point

 Purpose: decompress precisely one frame

 On entry:
   r0 to r5 are set up by Player as follows:
   r0 - source byte pointer
   r1 - output pointer - save output pixels here (word aligned)
   r2 - previous output pointer (allows copying from previous frame)
   r3 - pixel dither lookup table
   r4 - return address (can't be r14...)
   r5..r12 - scratch
   r13 - small stack (RISC OS irq stack)
   r14 - unuseable
   processor mode: IRQ, interrupts ENABLED (usually)
   flags - irrelevant
 On exit
   r0 - next value of source byte pointer
   r1..r12 - irrelevant
   r13 - must be same value as on entry
   r14 - irrelevant
   processor mode: IRQ, interrupts ENABLED
   flags - irrelevant

 Decompressed pixels are stored as Words!!! For the MovingLines decompressor
 to work, the source pointer (r0) must be half word aligned.

The 'DecompresH' file
---------------------

A decompression provider can additionally provide a DecompresH program. This
has exactly the same interface as the Decompress file, but stores output
pixels as half words. This allows the Player program to use less store (and
run slightly faster) in some screen modes (in particular, mode 15).

Programs which use the decompression programs must not assume the existence
of 'DecompresH' forms of decompressors and must be capable of using the
'Decompress' file instead. For some compression formats, the gain in the
Player of using the half word output may be completely lost by inefficiency
in the decompressor.

The 'Decom24' file
-------------------

The 15 bit per pixel format allows fast colour lookup, but may not allow an
accurate representation of a colour. The 'Decom24' file has an identical
structure to the 'Decompress' file, but writes output colours as 24bpp
values (in words) in the original colour space (i.e. the colour space
conversion by table does not exist). Note that it is perfectly possible to
have a file with 24bpp values used in the video stream where the original
ARMovie file header still claims 16bpp, provided a 'Decompress' file is
also present (for example, see Decomp8).

The 'Decom24xxx' files
-------------------------------------

These files have the same interfaces as 'Decom24', but write their output
in the 24bpp RGB colour space. A three character suffix represents which
colour space the source comes from - e.g. 'Decom24YUV' or 'Decom24YIQ' -
and this suffix should also be present in the ARMovie header at the end of
the bits per pixel line. Note that no suffix is possible, defaulting back
to 'Decom24' which provides no colour space translation (and so had better
imply an RGB colour space source!).

NOTE: Decom24xxx writers - the target colour space is 8 bit R G B with each
value in the range 0..255 (off..on), assumed gamma corrected (i.e. they will
be presented directly to DACs and then to the CRT with no extra
correction/changes).

The Batch Compression interface
===============================

A program 'BatchComp' can be provided in the Decomp<N> directory. The
program should obey the compression parameters given below and take an
ARMovie file as source (for example, by using the Decompress interface) and
compress it to produce some of the basic building blocks of an ARMovie file
which will later be turned into a final ARMovie file using 'Join' [aside:
the mechanics of produce a multitrack ARMovie file are sufficiently complex
that it is best to have all the capability in one program]. BatchComp should
read a provided ARMovie Header file in order to work out what to do: it
can use the header file to determine the number of frames per second, the
number of frames per chunk and the volume of the sound data per chunk. It is
then able to calculate (based on the supplied latency and bandwidth of the
target decompression device) how many bytes it can allocate to each frame of
video data. Larger numbers of frames per chunk mean more tolerance of device
latency, but larger chunks mean the movie takes more memory to decompress
and takes longer to start playing (since it has to read the first chunk).
Headers should be set up for 2 second chunks by default.

CAUTION: BatchComp programs may only manage simple resampling to generate
movies at different frame rates from the source. For example,
MovingLine.BatchComp can make a 12.5 fps movie from a 25 fps source (or a 30
fps one from a 60 fps) but not make 12.5 from 30 (or 60) [and obviously not
make 25 from 12.5...]

BatchComp writes video chunks and optionally key frames (key frames permit
the video data to be started from any point in the file, but they take up
space, being uncompressed 16bit per pixel data, so if the file only needs to
play from the start, or the video decompressor does not need key frames they
can be omitted). BatchComp writes all data in the specified destination
directory (which is where the the ARMovie header file is, called "Header")
and the video chunks will be filed separately using the following name for
each chunk:

 "Images"+STR$(chunk%DIV77)+"."+RIGHT$("0"+STR$(chunk%MOD77),2)

If writing key frames they are filed as:

 "Keys"+STR$(chunk%DIV77)+"."+RIGHT$("0"+STR$(chunk%MOD77),2)

These data files form part of the input data for the 'Join' program which
is used to make an ARMovie file after the compressor has run.

BatchComp may write a log of all the work it has done, this is stored as Log
in the destination, and may write similar information to the screen (the
Acorn compressors do). If the BatchComp process is slow (it is for the Acorn
software compressors), then BatchComp should periodically (e.g. 20 minute
intervals) save enough data so that it can continue in case the computer
stops unexpectedly: (Acorn compressors save this is in three files
"StoppedD", "StoppedC" and "StoppedF" in the destination directory - for
the correct working of !AREncode it is recommended that the files begin
with "Stopped").

Similarly, it is recommended that BatchComp be suspendable such that it can
be restarted. Pressing the space bar should be used as the trigger for this
so that your BatchComp behaves that same as other BatchComps. It is expected
that a similar set of Stopped files be used for restarting BatchComp some
time later.

It can be convenient for several collections of header/images/keys to be
stored in the same directory: the compressor parameter -index N allows any
number of collections to be stored with names Header/Images/Keys/Log
(default) 1Header/1Images/1Keys/1Log etc. Stopped files are also prefixed by
the index.

If a compression program has a BASIC shell (or is written entirely in BASIC)
then there is a common front end library (<ARMovie$Dir>.Tools.CompLib) which
provides much of the mechanics for writing a BatchComp. The Moving Lines and
Moving Blocks compressors use this.

The parameters are:

-arm2      limit the size of a frame to something that can be decompressed
           by an 8MHz ARM2 machine, if this has any meaning for that
           decompressor.

-batch     set batch mode: BatchComp should not attempt to detect the space
           bar being pressed to suspend it. This is to stop command files
           being sucked in a byte at a time each time a detect for space bar
           being pressed was made.

-dest b.   set directory for destination (compressed) data: the Header, Images,
           Keys are here. If a full pathname is used for this and the source,
           then the BatchComp program should be able to continue processing
           whatever else is happening (it may be multitasking in a task
           window for example). [aside: if the destination doesn't end in a
           directory separator, then add one!]

-dirty     specifies a 'quick and dirty' approach to compression if
           possible. For example, in the MovingLines compressor it tells the
           compressor to use a faster but less accurate method of varying
           the quality to attain the target frame size. Parameter is ignored
           by compressors which can attach no meaning to this hint.

-display N causes BatchComp to switch to screen mode N and display the
           images being compressed. Typically the original image would be
           displayed on the bottom right of the screen and the compressed
           image on the bottom left. If the compressed image can't be fitted
           then only the uncompressed image would be displayed. N may be a
           RISC OS 3.5 mode description string. Using this option when
           invoking BatchComp from a TaskWindow is not recommended!

-double    assume the decompressor will be using double buffered data
           (default: single) when it plays the movie back: this makes the
           number of bytes available for a frame larger for devices with
           high latency (such as CDROM) and thus improves the quality of the
           movie. The only result of asserting this is to change the text
           printed by the compressor when it calculates the amount of
           memory needed to decompress the movie (which is, of course,
           specific to a codec).

-filter    invoke filter or list of filters separated by ';' characters.
           Filters are stored in directory <ARMovie$Dir>.Tools.Filters.
           Filters are applied in the specified order:

              -filter TClamp;Median

           applies the temporal clamp first, then the spatial median. See
           below for filter machine code interface.

-index N   select different Header, Images, Keys, Log combination (default
           is no prefix). Different header files can be used to make
           different movies from the same source - for example with
           different sound tracks or with different frame per second rates.

-nokeys    prevents the key frames being written: this results in a smaller
           movie file which can only be started at the beginning (or at
           other "natural" cut points - for example where the movie changes
           from all white to all black). If the key frame data structure was
           written, a movie without keys can still be made if required.
           If keys are not important to the video compression algorithm, then
           this parameter is ignored.

-notext    suppresses all textual output

-quality N set quality mode and the quality: in the normal mode, BatchComp
           programs adjust the quality to maintain the average frame size at
           the required value. However, this may take time and can produce a
           movie with frames of poor quality. Using -quality will produce a
           movie with a particular quality for all frames, but the frame
           size varies. Quality numbers are higher for poor quality frames,
           lower for frames which are very accurately matched to the source
           frames. Users should make a small test run to find a suitable
           quality value for a particular movie and compression program -
           values for different Decomp<N>.BatchComp for a similar visual
           quality may well be very different (and produce different amounts
           of output). Quality 0 is taken to mean IDENTICAL with the source,
           there is no specified upper limit (users should be referred to
           information on the values for a particular BatchComp).

-restart   continue processing using the "Stopped" files. Only -dest,
           -index and -source need be given. It is recommended that the
           StoppedD file be used to store all the other parameters.
           New values for -batch and -display are required.

-size N    set frame size to N (may come from a data/latency computation).
           Values in the range 5000 to 10000 are reasonable for playback
           from hard discs.

-source f  set name of source file

-startat N set initial start point from the source file in centiseconds
           (default zero). (just like -startat when playing a movie).

Any not understood parameters should be ignored. Parameter strings may have
got too long for the RISC OS 256 character limit on command strings, so some
of the parameters may be in 'ARMovie$ExtraArgs': if this exists and has a
non-zero length, then it should also be read for input parameters, then set
to a null string. Programmers are encouraged to read the
MovingLine.BatchComp source...

Note that it is possible to make 'BatchComp' a directory and embed more
resources within it, the entry BatchComp.!Run being used automatically
by the operating system.

The calculations to go from bandwidth/latency to frame size are:

  size=((chunktime*double-latency)/double*datarate-soundbytes ANDNOT1023)/fpf

where the variables are:

chunktime: time for a chunk           (e.g. 2 seconds)
double:    1 for single buffer
           2 for double buffer
latency:   latency time               (e.g. 0.4 seconds)
datarate:  data rate for time base    (e.g. 150*1024 bytes per second)
soundbytes:number of bytes of sound data in a chunk
fpf:       frames per chunk

The soundbytes value is calculated from reading all the sound track
information in the header. Chunktime and fpf values are also from the
header.

The program should claim as much application workspace (only) as it requires.

ARMovie$CompressionStatus is an OS variable that the compressor writes to as
it exits (for whatever reason). It can take the following values:

 - does not exist: no error, job successful

 - exists and contains SUSPEND: job suspended, new stopped files written
   for continuation later.

 - else: contains textual error (reason why BatchComp stopped). The compression
   job has been aborted 'due to unforeseen circumstances'. If stopped files
   are present it can be continued (the files may be historical!)

The value must be read and remembered by the calling program on the exit from
BatchComp - it may be reset by any other compression tasks which are floating
around the machine when the next task swap occurs.

Filters
-------

Image processing filters can make a big difference to the quality of compressed
scenes. If dithering has been used, however, the filters may dramatically
reduce the colour range. It is also possible to use the filter code in other
programs. The interface is as follows:

Callers load the filter code at a quad word aligned address in memory. The init
routine is called. Then the filter routine is called as necessary.

First word: init entry point
(byte offset 0)

 Purpose: allows filter to initialise any values it needs (these are stored
          inside the filter's code space or allocated by the init routine).

 On entry:
   r0 - source x size of movie
   r1 - source y size of movie
   r2 - colour space of movie: 0: RGB 5,5,5
                               1: YUV 5,5,5
                               2: Y 8
                               3: RGB 8,8,8
                               4: YUV 8,8,8
   r3..r12 - undefined: scratch
   r13 - stack
   r14 - return address
   processor mode: USR
 On exit
   r0-r12 - may be corrupted
   r13 - same as on entry
   flags - V set signals error: r0 points to an error block

Second word: filter one frame

 Purpose: apply the filter and produce one filtered frame

 On entry:
   r0 - buffer to be filtered (word aligned)
   r1 - previous output pointer (allows temporal filtering from previous frame)
        (0 if no previous frame available)
   r2 - result pointer - save output pixels here (word aligned)
   r3..r12 - scratch
   r13 - stack
   r14 - return address
   processor mode: USR *or* SVC, interrupts ENABLED (usually)
   flags - irrelevant
 On exit
   r0..r12 - irrelevant
   r13 - must be same value as on entry
   r14 - irrelevant
   processor mode: same as called mode
   flags - irrelevant

 Pixels are stored as Words!!! (well, the bottom 15 bits of one, anyway)